Meta-Model Creation

MetaModels can be created directly from other core models using the create_metamodel_from_design or create_metamodel_from_data methods of a core model, or by using the create_metamodel() function, which can create a MetaModel directly from a scope and experimental results, without requiring a core model instance. Each of these functions returns a PythonCoreModel that already wraps the MetaModel in an interface ready for use with other TMIP-EMAT tools, so that in typical cases the user does not need to interact with or know anything about the MetaModel class itself, unless they care to dive in to the underlying core or mathematical structures.

emat.create_metamodel(scope, experiments: pandas.core.frame.DataFrame, metamodel_id: int = None, db=None, include_measures=None, exclude_measures=None, random_state=None, experiment_stratification=None, suppress_converge_warnings=False, regressor=None, name=None)

Create a MetaModel from a set of input and output observations.

Parameters:

• experiments (pandas.DataFrame) – This dataframe should contain all of the experimental inputs and outputs, including values for each uncertainty, level, constant, and performance measure.
• metamodel_id (int, optional) – An identifier for this meta-model. If not given, a unique id number will be created randomly (if not db is given) or sequentially based on any existing metamodels already stored in the database.
• db (Database, optional) – The database to use for loading and saving metamodels. If none is given here, the metamodel will not be stored in a database otherwise, the metamodel is automatically saved to the database after it is created.
• include_measures (Collection[str], optional) – If provided, only output performance measures with names in this set will be included.
• exclude_measures (Collection[str], optional) – If provided, only output performance measures with names not in this set will be included.
• random_state (int, optional) – A random state to use in the metamodel regression fitting.
• experiment_stratification (pandas.Series, optional) – A stratification of experiments, used in cross-validation.
• suppress_converge_warnings (bool, default False) – Suppress convergence warnings during metamodel fitting.
• regressor (Estimator, optional) – A scikit-learn estimator implementing a multi-target regression. If not given, a detrended simple Gaussian process regression is used.
• name (str, optional) – A descriptive name for this metamodel.

Returns:

a callable object that, when called as if a function, accepts keyword arguments as inputs and returns a dictionary of (measure name: value) pairs.

Return type:

PythonCoreModel

To demostrate the creation of a meta-model, we will use the Road Test example model included with TMIP-EMAT. We will first create and run a design of experiments, to have some experimental data to define the meta-model.

import emat.examples
scope, db, model = emat.examples.road_test()
design = model.design_experiments(design_name='lhs')
results = model.run_experiments(design)

We can then create a meta-model automatically from these experiments.

mm = model.create_metamodel_from_design('lhs')
mm

<emat.PythonCoreModel "EMATMeta", metamodel_id=1 with 2 constants, 7 uncertainties, 4 levers, 7 measures>

If you are using the default meta-model regressor, as we are doing here, you can directly access a cross-validation method that uses the experimental data to evaluate the quality of the regression model. The cross_val_scores provides a measure of how well the meta-model predicts the experimental outcomes, similar to an measure on a linear regression model.

mm.cross_val_scores()

no_build_travel_time          0.996413
build_travel_time             0.982765
time_savings                  0.929636
value_of_time_savings         0.821943
net_benefits                  0.589034
cost_of_capacity_expansion    0.934528
present_cost_expansion        0.949307
dtype: float64

We can apply the meta-model directly on a new design of experiments, and use the contrast_experiments visualization tool to review how well the meta-model is replicating the underlying model’s results.

design2 = mm.design_experiments(design_name='lhs_meta', n_samples=5000)
results2 = mm.run_experiments(design2)

from emat.analysis import contrast_experiments
contrast_experiments(mm.scope, results2, results)

No Build Time

Build Time

Time Savings

Value Time Save

Net Benefits

Cost of Expand

Present Cost